Compositional Verification for Autonomous Systems with Deep Learning Components

As autonomy becomes prevalent in many applications, ranging from recommendation systems to fully autonomous vehicles, there is an increased need to provide safety guarantees for such systems. The problem is difficult, as these are large, complex systems which operate in uncertain environments, requiring data-driven machine-learning components. However, learning techniques such as Deep Neural Networks, widely used today, are inherently unpredictable and lack the theoretical foundations to provide strong assurance guarantees. We present a compositional approach for the scalable, formal verification of autonomous systems that contain Deep Neural Network components. The approach uses assume-guarantee reasoning whereby {\em contracts}, encoding the input-output behavior of individual components, allow the designer to model and incorporate the behavior of the learning-enabled components working side-by-side with the other components. We illustrate the approach on an example taken from the autonomous vehicles domain

Modeling and Solving the Rush Hour puzzle

We introduce the physical puzzle Rush Hour and its generalization. We briefly survey its complexity limits, then we model and solve it using declarative paradigms. In particular, we provide a constraint programming encoding in MiniZinc and a model in Answer Set Programming and we report and compare experimental results. Although this is simply a game, the kind of reasoning involved is the same that autonomous vehicles should do for exiting a garage. This shows the potential of logic programming for problems concerning transport problems and self-driving cars

Towards Analyzing Semantic Robustness of Deep Neural Networks

Despite the impressive performance of Deep Neural Networks (DNNs) on various vision tasks, they still exhibit erroneous high sensitivity toward semantic primitives (e.g. object pose). We propose a theoretically grounded analysis for DNN robustness in the semantic space. We qualitatively analyze different DNNs' semantic robustness by visualizing the DNN global behavior as semantic maps and observe interesting behavior of some DNNs. Since generating these semantic maps does not scale well with the dimensionality of the semantic space, we develop a bottom-up approach to detect robust regions of DNNs. To achieve this, we formalize the problem of finding robust semantic regions of the network as optimizing integral bounds and we develop expressions for update directions of the region bounds. We use our developed formulations to quantitatively evaluate the semantic robustness of different popular network architectures. We show through extensive experimentation that several networks, while trained on the same dataset and enjoying comparable accuracy, do not necessarily perform similarly in semantic robustness. For example, InceptionV3 is more accurate despite being less semantically robust than ResNet50. We hope that this tool will serve as a milestone towards understanding the semantic robustness of DNNs.Comment: Presented at European conference on computer vision (ECCV 2020) Workshop on Adversarial Robustness in the Real World ( https://eccv20-adv-workshop.github.io/ ) [best paper award]. The code is available at https://github.com/ajhamdi/semantic-robustnes

Compositional Falsification of Cyber-Physical Systems with Machine Learning Components

Cyber-physical systems (CPS), such as automotive systems, are starting to include sophisticated machine learning (ML) components. Their correctness, therefore, depends on properties of the inner ML modules. While learning algorithms aim to generalize from examples, they are only as good as the examples provided, and recent efforts have shown that they can produce inconsistent output under small adversarial perturbations. This raises the question: can the output from learning components can lead to a failure of the entire CPS? In this work, we address this question by formulating it as a problem of falsifying signal temporal logic (STL) specifications for CPS with ML components. We propose a compositional falsification framework where a temporal logic falsifier and a machine learning analyzer cooperate with the aim of finding falsifying executions of the considered model. The efficacy of the proposed technique is shown on an automatic emergency braking system model with a perception component based on deep neural networks

Formal Verification of Neural Network Controlled Autonomous Systems

In this paper, we consider the problem of formally verifying the safety of an autonomous robot equipped with a Neural Network (NN) controller that processes LiDAR images to produce control actions. Given a workspace that is characterized by a set of polytopic obstacles, our objective is to compute the set of safe initial conditions such that a robot trajectory starting from these initial conditions is guaranteed to avoid the obstacles. Our approach is to construct a finite state abstraction of the system and use standard reachability analysis over the finite state abstraction to compute the set of the safe initial states. The first technical problem in computing the finite state abstraction is to mathematically model the imaging function that maps the robot position to the LiDAR image. To that end, we introduce the notion of imaging-adapted sets as partitions of the workspace in which the imaging function is guaranteed to be affine. We develop a polynomial-time algorithm to partition the workspace into imaging-adapted sets along with computing the corresponding affine imaging functions. Given this workspace partitioning, a discrete-time linear dynamics of the robot, and a pre-trained NN controller with Rectified Linear Unit (ReLU) nonlinearity, the second technical challenge is to analyze the behavior of the neural network. To that end, we utilize a Satisfiability Modulo Convex (SMC) encoding to enumerate all the possible segments of different ReLUs. SMC solvers then use a Boolean satisfiability solver and a convex programming solver and decompose the problem into smaller subproblems. To accelerate this process, we develop a pre-processing algorithm that could rapidly prune the space feasible ReLU segments. Finally, we demonstrate the efficiency of the proposed algorithms using numerical simulations with increasing complexity of the neural network controller

Reflection on multilayer mirrors beam profile and coherence properties

The main advantage of Bragg reflection from a multilayer mirror as a monochromator for hard X rays, is the higher photon flux density because of the larger spectral bandpass compared with crystal lattice reflection. The main disadvantage lies in the strong modulations of the reflected beam profile. This is a major issue for micro imaging applications, where multilayer based monochromators are frequently employed to deliver high photon flux density. A subject of particular interest is the origin of the beam profile modifications, namely the irregular stripe patterns, induced by the reflection on a multilayer. For multilayer coatings in general it is known that the substrate and its surface quality significantly influence the performance of mirrors, as the coating reproduces to a certain degree the roughness and shape of the substrate. This proceedings article reviews recent experiments that indicate potential options for producing wave front preserving multilayer mirrors, as well as new details on the particular mirrors our group has extensively studied in the pas

Verisig: verifying safety properties of hybrid systems with neural network controllers

This paper presents Verisig, a hybrid system approach to verifying safety properties of closed-loop systems using neural networks as controllers. We focus on sigmoid-based networks and exploit the fact that the sigmoid is the solution to a quadratic differential equation, which allows us to transform the neural network into an equivalent hybrid system. By composing the network’s hybrid system with the plant’s, we transform the problem into a hybrid system verification problem which can be solved using state-of-theart reachability tools. We show that reachability is decidable for networks with one hidden layer and decidable for general networks if Schanuel’s conjecture is true. We evaluate the applicability and scalability of Verisig in two case studies, one from reinforcement learning and one in which the neural network is used to approximate a model predictive controller

Determination of ππ\pi\pi scattering lengths from measurement of π+π−\pi^+\pi^- atom lifetime

The DIRAC experiment at CERN has achieved a sizeable production of $\pi^+\pi^-$ atoms and has significantly improved the precision on its lifetime determination. From a sample of 21227 atomic pairs, a 4% measurement of the S-wave $\pi\pi$ scattering length difference $|a_0-a_2| = (.0.2533^{+0.0080}_{-0.0078}|_\mathrm{stat}.{}^{+0.0078}_{-0.0073}|_\mathrm{syst})M_{\pi^+}^{-1}$ has been attained, providing an important test of Chiral Perturbation Theory.Comment: 6 pages, 6 figure

Archival processes of the water stable isotope signal in East Antarctic ice cores

The oldest ice core records are obtained from the East Antarctic Plateau. Water isotopes are key proxies to reconstructing past climatic conditions over the ice sheet and at the evaporation source. The accuracy of climate reconstructions depends on knowledge of all processes affecting water vapour, precipitation and snow isotopic compositions. Fractionation processes are well understood and can be integrated in trajectory-based Rayleigh distillation and isotope-enabled climate models. However, a quantitative understanding of processes potentially altering snow isotopic composition after deposition is still missing. In low-accumulation sites, such as those found in East Antarctica, these poorly constrained processes are likely to play a significant role and limit the interpretability of an ice core's isotopic composition.By combining observations of isotopic composition in vapour, precipitation, surface snow and buried snow from Dome C, a deep ice core site on the East Antarctic Plateau, we found indications of a seasonal impact of metamorphism on the surface snow isotopic signal when compared to the initial precipitation. Particularly in summer, exchanges of water molecules between vapour and snow are driven by the diurnal sublimation–condensation cycles. Overall, we observe in between precipitation events modification of the surface snow isotopic composition. Using high-resolution water isotopic composition profiles from snow pits at five Antarctic sites with different accumulation rates, we identified common patterns which cannot be attributed to the seasonal variability of precipitation. These differences in the precipitation, surface snow and buried snow isotopic composition provide evidence of post-deposition processes affecting ice core records in low-accumulation areas

Observation of the Cabibbo-suppressed decay Xi_c+ -> p K- pi+

We report the first observation of the Cabibbo-suppressed charm baryon decay Xi_c+ -> p K- pi+. We observe 150 +- 22 events for the signal. The data were accumulated using the SELEX spectrometer during the 1996-1997 fixed target run at Fermilab, chiefly from a 600 GeV/c Sigma- beam. The branching fractions of the decay relative to the Cabibbo-favored Xi_c+ -> Sigma+ K- pi+ and Xi_c+ -> X- pi+ pi+ are measured to be B(Xi_c+ -> p K- pi+)/B(Xi_c+ -> Sigma+ K- pi+) = 0.22 +- 0.06 +- 0.03 and B(Xi_c+ -> p K- pi+)/B(Xi_c+ -> X- pi+ pi+) = 0.20 +- 0.04 +- 0.02, respectively.Comment: 5 pages, RevTeX, 3 figures (postscript), Submitted to Phys. Rev. Let